The State of the Art in Image and Video Retrieval

Image and video retrieval continues to be one of the most exciting and fastest-growing research areas in the field of multimedia technology. What are the main challenges in image and video retrieval? Despite the sustained efforts in the last years, we think that the paramount challenge remains bridging the semantic gap. By this we mean that low level features are easily measured and computed, but the starting point of the retrieval process is typically the high level query from a human. Translating or converting the question posed by a human to the low level features seen by the computer illustrates the problem in bridging the semantic gap. However, the semantic gap is not merely translating high level features to low level features. The essence of a semantic query is understanding the meaning behind the query. This can involve understanding both the intellectual and emotional sides of the human, not merely the distilled logical portion of the query but also the personal preferences and emotional subtones of the query and the preferential form of the results. Another important aspect is that digital cameras are becoming widely available. The combined capacity to generate bits of these devices is not easy to express in ordinary numbers. And, at the same time, the growth in computer speed, disk capacity, and most of all the rapid expansion of the web will export these bits to wider and wider circles. The immediate question is what to do with all the information. One could store the digital information on tapes, CD-ROMs, DVDs or any such device but the level of access would be less than the well-known shoe boxes filled with tapes, old photographs, and letters. What is needed is that the techniques for organizing images and video stay in tune with the amounts of information. Therefore, there is an urgent need for a semantic understanding of image and video. Creating access to still images is still hard problem. It requires hard work, precise modeling, the inclusion of considerable amounts of a priori knowledge and solid experimentation to analyze the contents of a photograph. Luckily, it can be argued that the access to video is somehow a simpler problem than access to still images. Video comes as a sequence, so what moves together most likely forms an entity in real life, so segmentation of video is intrinsically simpler than